The ability to perceive two-dimensional images in three-dimensions by way of numerous different technologies has grown quite popular over the past few years. Providing an aspect of depth to two-dimensional images potentially creates a greater sense of realism to any depicted scene. This introduction of three-dimensional visual representation has greatly enhanced viewer experiences, especially in the realm of video games.
A number of techniques exist for the three-dimensional rendering of a given image. Most recently, a technique for projecting a two-dimensional image(s) into three-dimensional space known as depth-image-based rendering (DIBR) has been proposed. In contrast to former proposals, which often relied on the basic concept of “stereoscopic” video, i.e., the capturing, transmission, and display of two separate video streams—one for the left eye and one for the right eye—, this new idea is based on a more flexible joint transmission of monoscopic video (i.e., single video stream) and associated per-pixel depth information. From this data representation, one or more “virtual” views of the 3-D scene can then be generated in real-time at the receiver side by means of so-called DIBR techniques. This new approach to three-dimensional image rendering presents several advantages over previous approaches.
First, this approach allows 3-D projection or display to be adjusted to fit a wide range of different stereoscopic displays and projection systems. Because the required left—and right-eye views are only generated at the 3D-TV receiver, their appearance in terms of ‘perceived depth’ can be adapted to the particular viewing conditions. This provides the viewer with a customized 3-D experience that is comfortable to watch on any kind of stereoscopic or autostereoscopic 3D-TV display.
DIBR also allows for 2D-to-3D conversion based on “structure from motion” approaches that can be used to generate the required depth information for already recorded monoscopic video material. Thus, 3D video can be generated from 2D video for a wide range of programming, which could play a significant role in the success of 3D-TV.
Head motion parallax (i.e., apparent displacement or difference in the perceived position of an object caused by change in viewing angle) can be supported under DIBR to provide an additional extrastereoscopic depth cue. This eliminates the well-known “shear-distortions” (i.e., stereoscopic image appears to follow the observer when the observer changes viewing position) that are usually experienced with stereoscopic- or autostereoscopic 3D-TV systems.
Furthermore, photometrical asymmetries, e.g., in terms of brightness, contrast or color, between the left- and the right-eye view, which can destroy the stereoscopic sensation, are eliminated from the first, as both views are effectively synthesized from the same original image. Also, it enables automatic object segmentation based on depth-keying and allows for an easy integration of synthetic 3D objects into “real-world” sequences.
Lastly, this approach allows the viewer to adjust the reproduction of depth to suit his/her personal preferences—much like every conventional 2D-TV set allows the viewer to adjust the color reproduction by means of a (de-)saturization control. This is a very important feature because there is a difference in depth appreciation over age groups. A recent study by Norman et al., for example, demonstrated that older adults were less sensitive than younger adults to perceiving stereoscopic depth.
While each viewer may have a unique set of preferred depth settings, so too does each scene presented to the viewer. The content of each scene dictates what range of depth settings should be used for optimal viewing of the scene. One set of re-projection parameters may not be ideal for every scene. For example, different parameters may work better depending how much of the distant background is in the field of view. Because the content of a scene changes each time a scene changes, existing 3D systems do not take the content of a scene when determining re-projection parameters.
It is within this context that the embodiments of the present invention arise.